Web Survey Bibliography
Relevance & Research Question: Online questionnaires are a common utility for research companies. While survey software products offer many features, respondent fraud and indifferent or inattentive respondent behaviour remain critical issues. How can responses with such bad quality be identified in an automated process?
Methods & Data: The author proposes a post-fieldwork approach which is based on behaviour pattern detection and does not rely on control or trap questions. Using response quality indicators as well as discriminant analysis, logistic regression and an optional flag variable, responses are classified with regard to their quality. The 17 indicators focus on aspects such as response differentiation to open-ended questions, the time spent for answering the survey and monotonous behaviour in response to matrix questions. For the procedure to work, the survey should include open-ended questions, several matrix questions as well as a minimum of ten questions overall. An incentivized survey containing quality-related trap questions and other control measures is sent out to a Facebook river sample (n = 134) as well as a commercial panel sample (n = 1,000). This survey is used to generate a standard classification. Another five survey data sets from past real-case projects are finally used to examine the effectiveness of the procedure developed (157 <= n <= 2,603). R is used for calculating indicators, SPSS for discriminant and regression analysis. The automation process is specifically designed for QuestBack EFS software.
Results: Depending on the data, the procedure identifies between 2.5 and 5.2 per cent of all respondents as low-quality respondents. Judging from their indicator values, their behaviour is clearly suspected to indicate bad quality. Therefore they should be considered to be removed from the sample.
Added Value: The approach offers straight-forward ways of judging whether survey responses should be considered trustworthy in comparison to one another. This knowledge supports post-fieldwork data cleansing and reduces effects of distortion by low-quality data. The procedure is ready for implementation in the EFS software.
Web survey bibliography (4086)
- Facebook as a Tool for Respondent Tracing; 2015; Schneider, S. J., Burke-Garcia, A., Thomas, G.
- Social Science Survey Methodology Training: Understanding the Past and Assessing the Present to Shape...; 2015; Jans, M., Meyers, M., Fricker, S.
- Internet Research in Psychology; 2015; Gosling, S. D., Mason, W.
- Handbook of Health Survey Methods; 2015; Johnson, T. P. (Ed.)
- Adapting an interviewer - administered survey for web self - completion in a mixed - mode design ; 2015; Betts, P.; Cubbon, B.
- Future Training of Survey Methodologists; 2015; Kolenikov, S., Jans, M., O'Hare, B. C., Fricker, S.
- Automatic data collection on the Internet (web scraping); 2015; Boettcher, I.
- The Impact of Survey Mode (Mail versus Telephone) and Asking About Future Intentions; 2015; Beebe, T. J.
- Offline recruiting of young people for an online survey - what affects response rates; 2015; Zeglovits, E.
- Finding Item Nonresponse Patterns: Three Internet Survey Experiments Into the Effects of Nonresponse...; 2015; Van De Maat, J.
- Placement of the Linkage Consent Question in a Web Survey of Establishments; 2015; Sakshaug, J. W.; Vicari, B.
- The effectiveness of incentives on recruitment and retention rates: an experiment in a web survey; 2015; Mulder, J.; Douhou, S.
- Using WhatsApp as a Survey Tool; 2015; Ongena, Y. P.; Haan, M.
- The Effects of Adding a Mobile-Compatible Design to the American Life Panel; 2015; Toepoel, V.; Lugtig, P. J.; Amin, A.
- Technology and Reporting of Daily Activities – Considerations for Analysis of Behaviours in Mixed...; 2015; Fisher, K.; Gershuny, J.
- Does the Use of Mobile Devices (Tablets and Smartphones) Affect Survey Quality and Choice Behaviour...; 2015; Glenk, K.; Liebe, U.; Oehlmann, M.
- Smartphones @work; 2015; Bittman, M.
- Measurement Error in Discontinuous Online Survey Panels: Panel Conditioning and Data Quality; 2015; Atkeson, L. R.; Adams, A. N.; Karp, J. A.
- Cheating in web surveys. Evidence from a split-ballot repeated experiment on knowledge questions on...; 2015; Ladini, R.; Vezzoni, C.
- Does Personalized Feedback Increase Respondent Motivation?; 2015; Kroh, M.; Kuhne, S.
- Adapting Grid Questions for Mobile Devices; 2015; de Bruijne, M.; Das, M.; van Soest, A.; Wijnant, A.
- Unplanned use of mobile devices in a probabilistic online panel survey: Patterns of use and implications...; 2015; Poggio, T.; Bosnjak, M.; Bandilla, W.; Weyandt, K.
- The importance of scale direction between different modes; 2015; Agalioti-sgompou, V.
- Impact of response scale direction on survey responses in web and mobile web surveys; 2015; Yan, T.; Keusch, F.
- Comparing response order experiments with probability and non-probability samples; 2015; Yeager, D. S.; Krosnick, J. A.; Silber, H.
- Direction of Response Format in Web and Paper & Pencil Surveys; 2015
- Comparison of different mixed-mode and face - to face surveys - response rates and costs; 2015; Ainsaar, M.; Hendrikson, R.
- Nonresponse and Measurement Bias in Web surveys ; 2015; Metzler, A.; Fuchs, M.
- Correlates of early and late responses to surveys in an online panel; 2015; Douhou, S.; Vis, C.
- Higher Item Nonresponse Rates Caused by Slider Scales in Web Surveys; 2015; Toepoel, V.; Funke, F.
- The effect of response formats on data quality and comparability across online PC and smartphone surveys...; 2015; Cleary, A.; Allum, N.; Kolbas, V.
- Mobile devices in a web panel: what are the results of adjusting questionnaires for smartphones and...; 2015; de Bruijne, M.; Wijnant, A.
- Online Eye-Tracking of Dynamic Advertising Content in (Mobile) Web-Surveys; 2015; Berger, S.
- Deep impact or no impact, evaluating opportunities for a new question type: Statement allocation on...; 2015; Schmidt, S.
- Approaches for Evaluating Online Survey Response Quality; 2015; Gluck, N.
- Coding Surveys on their Item Characteristics: Reliability Diagnostics; 2015; Bais, F.; Schouten, B.; Toepoel, V.
- Predicting Response Times in Web Surveys; 2015; Wenz, A.
- Positioning of Clarification Features in Open Frequency and Open Narrative Questions; 2015; Fuchs, M.; Metzler, A.
- The Role of Device Type and Respondent Characteristics in Internet Panel Survey Breakoff; 2015; McCutcheon, A. L.
- Web Survey Invitations: Design Features to Improve Response Rates; 2015; Hughes, J.; Marlar, J.
- Advance Postcard Mailing Improves Web Panel Survey Participation; 2015; Bertoni, N.; Burkey, A.; Caldaro, M.; Keeter, S.; DiSogra, C.; McGeeney, K.
- Mobile Devices for the Collection of Sensitive Information; 2015; Maitland, A.; Mercer, A. W.; Tourangeau, K.; Williams, Do.
- What Is The Impact of Smartphone Optimization on Long Surveys?; 2015; Cole, J.; Brooks, K.; Sarraf, S.
- Examining the Impact of Mobile First and Responsive Web Design on Desktop and Mobile Respondents; 2015; Tharp, D.
- Can An Importance Prompt Reduce Item Nonresponse For Demographic Items Across Web and Mail Modes?; 2015; Israel, G. D.
- Leveraging Area Probability Sampling in Recruiting Households for Web Surveys; 2015; Copeland, K.; Pedlow, K.; Tupek, A.
- Reducing Coverage Error in a Web Survey of College Students; 2015; Daley, K.; Pacer, J.
- Influences on Response Latency in a Web Survey; 2015; Ackermann, A.; Cheng, H. W.; Howard Ecklund, E.; Kolenikov, S.; Phillips, B. T.
- App vs. Web for Surveys of Smartphone Users; 2015; Igielnik, R.; McGeeney, K.
- Where Does the Platform Matter: The Impact of Geographic Clustering in Device Ownership and Internet...; 2015; Bilgen, I.; English, N.; Stern, M. J.; Ventura, I.